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ABSTRACT 

Although two-stage testing is not the most efficient 
form of adaptive testing, it has soma advantages. In this paper, 
linear programming models are given for the construction of two-stage 
tests. In these models, practical constraints with respect to, among 
other things, test composition, administration time, emd inter-item 
dependencies play an important role. Research indicates that 
two-stage tests can I3e constructed ^th sequentially and 
simultaneously. Models based on the maximin model for test 
construction developed by W. J. van der Linden ?nd E. 
Boe)c)cooi-Timminga (1989) are formulated for the sequ^^ij^al case, with 
constraints specified at test and subtest levels. It is assumed that 
a banic of items calibrated under the item response model is available 
and that "Information" is used in accordance with G. H. Fischer's 
information model. The maximin design is used in order to select the 
items that maximize the information in the test, while the resulting 
test information function still has the desired shape. The paper 
concludes that simultaneous test construction has the disadvantage of 
having a large number of variables; hence ^ constraints must be 
considered. (TJH) 
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Abstract 

Although two-stage testing is not the most efficient form of 
adaptive testing, it has some advantages. In this paper 
linear programming models are given for the construction of 
two-stage tests. In these models practical constraints with 
respect to. e g.. test composition, administration time, 
inter— item dependencies, play an important role. Two— stage 
tests can be constructed both sequentially and 
simultaneously. Models are formulated for the sequential case 
with constraints specified at test and subtest level. 
Simultaneous test construction has the disadvantage that a 
large number of variables and constraints have to be 
considered. 
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The construction of two-stage tests 

Two-stage testing Implies that the examinee Is first 
confronted with a routing test. The choice of the second test 
depends on the score on the routing test. 

Two-stage tests will be most valuable In situations 
where the group tested has a range of ability too wide to be 
measxired effectively by a peaked conventional test (Lord, 
1980, p. 146). Other forms of adaptive testing are often more 
efficient than two-stage testing. However two-stage tests 
can be useful. Their advantages are that they can be 
administered by paper and pencil (Lord, 1980; Fischer & 
Pendl, 1980), and, under the Rasch model, that the ability 
estimates are easily computed (Fischer & Pendl, 1980; Glas, 
1988). Walner and Ilely (1987) give a number of problems that 
are associated with most forms of adaptive testing, like 
context effects, lack of robustness, and Item difficulty 
ordering (the Items administered In the beginning are too 
difficult for the least able students). In general, two-stage 
tests need not be sensitive to these problems If they are 
dealt with appropriately when constructing the test. 

As pointed out by Ten (1983) linear programming models 
(LP models) can be used for the construction of tests 
(although we use the word "model** an LP model Is In fact a 
problem) . LP models for the construction of two-stage tests 
will be given In this paper. In these models practical 
constraints. I.e., demands with respect to the properties of 
the test, are taken Into account. Practical constraints can. 
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for i iStance, be used to control the test coflsposltlon and the 
administration time. The oiodels are based on the maximin 
model for test construction (van der Linden & Boekkooi- 
Timminga, 1989). 

Further on, it is assumed that a bank of items 
calibrated under an item usponse model is available and when 
the word "information" is used Fisher's information is meant. 
In the maximin model the test constructor has to provide the 
relative shape of a target test informacion function by 
giving target values at certain points. The idea is to select 
tt^ items such that they maximize the information in the 
test, while the resulting test information function still has 
the desired shape. 

Define the decision variables Xj^ as follows: 



Let pi, 1 = 1 Lbe the relative amount of information 

that io required at ability level 6i and the amount of 

information at ability level for item i. The maximin model 
can then be formulated as follows: 




0 item i not in the test 



1 item i in the test. 



(1) 



Max. y 



subject to 
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<2) 



- PlY ^ 0, 1 = 1.2 L. 



(3) I x. = n. 

i=l ^ 



(4) Xi e {0.1>. 1=1,2 1. 

(5) Y ^ 0. 

where n Is the number of Items In the test. I Is the number 
of Items la the Item b€mk, and y Is a dummy variable. In this 
representation It Is clear that y can be considered a lower 
bound to the weighted sums of decisions variables 
Z^.^IiOi) p^^xt and that the values of Xi are selected such 
that this lower bound Is maximal. 

The maxlmln model Is In the operations research 
literature known as a mixed Integer linear programming model. 
These models can be solved by brcmch-and-bound methods. I.e.. 
methods are available that maximize y €uid slmultsuieously 

confute the corresponding optimal values for x^, 1 = 1 1 

(Land & Dolg. 1960). 

Practical Constraints 

The linear programming models for the construction of 
two-*stage tests may Include a number of practical constraints 
as formulated by van der Linden and Boekkool-Tlmmlnga (1989). 
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In most item banks we can distinguish subsets of items. Items 
can. for instance, be grouped in subsets on the basis of 
their content or format (e.g.. multiple choice, completion). 

Generally, three different kinds of subsets can be 
distinguished: 



Ej (J = 1. 2, .... J^): At most nj^ items may be selected 

from the subsets Ij; 
Ej (J = 1. 2 J^;): Exactly n^; items should be selected 

from the suhsets Ej : 
6j (J - 1. 2 Jq): At least uq items should be selected 

from the subsets Gj . 



The following constraints are related to the subsets given: 



(6) Z Xi i nja . 3^1.2 Jk. 



(7) Z Xi = nE.. J = 1. 2 Je. 

i€Ej J 



(8) Z^ Xi ^ noj. J = 1. 2 Jq. 



i€Gj 



For example, let ^a item bank for French be partitioned with 
respect to its content in a vocabulary, a grammar and a 
reading comprehension part and with respect to its format of 
items in a multiple-choice and a matching part. A test 
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constructor has the following demands with respect to the 
coaqposltlon of the test: 

1. The number of grammar items in the test is not greater 

than 5: 

I ^ 5, 

id: ^ 

where T.^ is the subset of graanar items, 

2. The number of vocabulary items in the test is equal to 
6: 

I = 6. 

i€Ei ^ 

irtiere is the subset of vocabulary items, 

3. The number of reading coaqprehension items in the test is 

greater than 5: 

£ X. i. 6. 
i€Gi ^ 

where 0^ is the subset of reading coooprehension items, 

4. The number of multiple choice items in the test is not 

greater than 12: 

£ X. s 12, 
i€K2 ^ 

where E2 the subset of multiple choice items, 

5. The number of matching items in the test is greater than 
6; 

I Xi i 7, 
l€G2 ^ 

where 63 is the subset of matching items. 
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If we also want to restrict the administration time of 
the test, we can do this by Including the following 
constraint : 

I 

(9) I tiXi i T, 
1=1 

whare tj^ Is an estimate of the time an examinee from the 
population needs for answering Item 1 and T Is the upper 
bound on the administration time for that test. 

Another possible kind of constraint reflects the 
dependencies among Items. It Is possible that the Item bank 

contains subsets of Items Vj (j = 1, 2 Jy) from which 

It Is not allowed to select more than one Item, because every 
Item In such a subset contains Information about the answers 
to the other Items In that subset. This demand can be 
formulated In a linear constraint as follows: 

(10) I Xi ^ 1. j = 1. 2 Jy. 

l.Vj 

It may also be desirable to select either all or none of 

the items from a subset (J = 1, 2 J^^ ) : 

(H) I Xi = |Wi|xi., i = 1. 2 Jw. 

ieWj 

where |Wj| is the number of items in Wj and Xj.^ is an 
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arbitrary item in Hj. 

Sequential Test Construction 

If a two-stage test is admiu. jed as a paper and 
pencil test, a decision mui- be made about the ability levels 
at which the second tests should aim. Theunissen (1986) shows 
how these ability levels can be computed, if the tests are 
constructed sequentially. 

If the test is administered by a r^omputer. it is 
possible to adapt the second test to the individual ability 
of the examinees, because then the ability of the ezamipees 
can be estimated using the score on the routing test. 
Therefore, we will base the selection of items for the second 
test on the estimated ability. This ioqplies that the subtests 
are constructed sequentially. 

k test constructor may wish to impose coL^traints on the 
item selection at two levels: the subtest or the test ^evel. 
Both possibilities will be considered in the following two 
sections . 

Constraints at Subtest Level 

In this case only constraints at subtest level are 
considered. A general LP model that selects items for the 
routing test r can be formulated as follows: 

(12) Max. yy 
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subject to 



I 

(13) I Ii(ei)Xir - Yr i 0. 1 = 1 L . 

i=l 



I 

(14) E Xir = nr. 
i=l 



(IS) Z Xir s nZrA- 3 = 1 Jr,- 



(16) I Xir = nEjj. j = 1 Je^. 



(17) 



I Xir i DQ^ j = 1 Jgj. 



I 

(18) Z tirXir s Tr. 

i=l 



(19) Z Xir s 1. j = 1 Jvr. 



(20) Z Xir = |Wrjl«ijr. 3 = 1 ^Wr' 

(21) Xir e {0,1}. i = 1 I . 

(22) Yr i 0- 
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The number of items in the routing test is equal to n^* In 
this model the values of p]^ are set to one (see the 
constraints in inequality 2). Because an adaptive test is 
supposed to measure accurately ovar the whole ability range, 
it is assumed that the test constructor wants the same amount 
of information for each ability level. The advantage of this 
approach is that the test constructor does not have to 
specify the relative amount of inforiLation that is required 

at 61, 1 = 1 L. 

In the case of coooputerized test administration the 
construction of the second test is based on the estimate O"^ 
of the/ ability on the routing test. If the test is 
administered as a paper and pencil test, then the number of 
ability levels has to be specified in advance. The second 
tests are constructed such that they give maximal information 
on the specified ability levels. A LP model for the 
construction of a second test s at ability level 8*^ is: 



(23) 



I 

Max. I Ii(e*)xis 
i=l 



subject to 



(24) 




(14) - (17) with subindex r replaced by s. 
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I 

(25) Z t%lg(e*)Xig ^ Tg, 

i=l 

(19) - (21) with sublndex r replaced by s, 

where U is the set of items selected for the routing test. It 
should be noted that G^j and Gg^ can be two totally different 
sets of items. In constraint (25) we suppose that the time 
needed for answering an item depends on the ability 6*. In 
pvactice, this implies that a large number of response times 
must be estimated. These €Stimates are often not available, 
and it will then not be possible to include constraint (25) 
in the model. On the other hand, the omission of constraint 
(25) will enable the use of single and efficient algorithms 
for solving the LP model (see Appendix). If siosple algorithms 
cannot be used we have to use heuristics. A heuristic based 
on the branch-and-bound method that can be used for the 
present test construction problem is given by Adema (1988). 

Constraints at Test Level 

If the test constructor specifies demands with respect 
to the item selection at the level of the test, then tne 
above specified models are no longer applicable. In this 
section models which can be used for the construction of two- 
stage tests with demands at test level are given. In these 
models the constraint on the administration time is omitted 
because this constraint can rarely be included in view of a 
lack of data. The LP m^del for the construction of the 
routing test r can be formulated as follows: 
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(26) Max. 7]. 
subject to 



(13) - (14). 



(30) 
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(27) S Xir s ng^. j = 1 JE' 

i€E^ J 



(28) 1 *Li ^ ^Za- J = 1 Jr. 



(29) X Xir s 1. j = 1 Jy. 

i€Vj 



^ «ir = |Wj|Xijr. J = 1 JW. 

(21) - (22). 



In the model it is supposed that both the number of items in 
the routing and the second tests are specified separately. 
The model shows that restrictions on the subsets 6j . 

j=l Jq are not needed. The way the constraints in (30) 

are formulated does not allow some of the items of Wj to be 
in the routing test while the others are in the second test. 
The model 2ot the second test is: 



I 

(31) Max. Z Ii(e*)Xis 
i=l 
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subject to 



I 

(32) 1^ Xis = ns. 



(33) I Xis = 0. 



(34) Z ^ - 2 xir. j = 1 Jr. 



(35) S Xis = - S Zir. 3 = 1 Je. 

i€Ej ^ i€Ej 



(36) Z Xis St - Z Xir. j = 1 Jq. 

i€Gj 3 i€Gj 



(37) Z Xis S 1 - S Xir. j = 1 Jv. 



(38) Z Xis = iWjl'i^s. j = 1 Jw. 



(39) Xis € <0.1>. i = 1 I . 

where 0 is the set of items selected for the routing test. In 
this model the Xir's are no longer variables, because they 
are fixed at the value 0 or 1 after the model for the routing 
test has been solved. The constraints (31) - (33), (38) and 
(39) can also be found in the LP model for the construction 
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of second tests with demands at subtest level. These 
constraints are given again for the sake of clarification. 

It Is possible that no feasible solution for the second 
test can be found. This problem can be approached as follows: 
Let the Integer variables zg^ (j = 1,...., Jg) and 
ZQj (j = 1, .... , Jq) denote the minimum number of Items from 

sets Ej (j = 1 Jg) and Gj (j = 1 Jq) to be 

selected for the second test. Several criteria can be found 
for partitioning an Item bank In subsets of Items. Let H 
denote the number of criteria. The coefficients ^mE^ ^mSj 
are now defined as follows: 



These variables and coefficients are Included In the 
constraints In (27) of the model for the routing test as well 
as In two new kind of constraints: 




0 If the Items are not In Ej because of criterion m 

1 If the Items are In Ej because of criterion m 




0 If the Items are not In Gj because of criterion m 

1 If the Items are In Gj because of criterion m. 



(40) 




i = 1 Je. 



(41) 



S Xir + ^ HGi . 

l€Gj J J 



3 = 1 



Jg. 
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(42) ^ *fflEjZEj + ^ T^8' m = 1 M. 

The constraints In (42) exclude the possibility that, for 
instance, at least 7 addition Items and exactly 4 subtraction 
Items must be selected for the second test, while the number 
of Items In the second test Is 10. 



Simultaneous Test Construction 



If the routing and the second test are constructed 
before they are administered, It Is possible to construct the 
subtests simultaneously Instead of sequentially. The choice 
of the ability levels of the second tests, however, is a new 
problem. In this paper we will suppose that the ability 
levels of the second tests can be selected in advance. This 
can be done safely, becevse item Information function? are 
continuous, well behaved smooth functions. If practical 
demands with respect to the item selection are specified at 
subtest level it Is much easier to construct the subtests 
sequentially. Therefore, only demands at the level of the 
test are considered. The model that will be given is not 
always practical, because It Includes a Icurge number of 
decision variables "ad constraints. We will return to this 
problem at the end of the section. The advantage of 
simultaneous test construction compared to sequential test 
construction is the better distribution of "good" and "bad" 
items over ^ i routing and second test. 
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Define the decision variables xj^p by: 



«ip = { 



0 item i not in subtest p 



1 item i in subtest p. 



p = 1 P 



The routing test is denoted by p = 1 . So there are P - 1 
second tests. In the model 7;^ is equal to the minimum of 



where 6* is the ability level at which second test p is 

peaked for p = 2 P. The objective function of the model 

is to maximize these minima simultaneously. Because the value 
of y^ respectively y^ is influenced by the number of items in 
the routing test respectively the second tests, we have to 
weight the variables y^ and ja ^7 1/i^r ^/^s respectively 
in the objective function. So if we assume that the items in 
the routing test and the second tests are equally iooportant. 
then we can formulate the following model: 

(43) Max. Yt/^t + Ys/^a 



ill 



for 1 = 1. 



. . . . L and is equal to the minimum of 
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subjttct to 



(44) 2 Il(ei)«il ^ 7r. 

i«l 



1 = 1 L. 



(45) 1^ Ii(e^)xip i Ya. 



P = 2 P. 



(46) Z xii = nr. 

1=1 



(47) 2 xip = Os- 

1=1 ^ 



P = 2 P. 



(48) S xii + Z xip S nr^. 



(49) 



(50) 



(51) 



Z xii + S Xip = nv 



Z Xil + Z Xip 2 OQ^ 



j = 1.. 

p = 2.. 

j = 1.. 
P = 2.. 



j = 1.. 
p = 2.. 

j = 1.. 
p = 2.. 



.. Jz ; 
.. P . 

.. JE : 
.. P . 



• . Jq; 
.. P . 

.. Jv: 
.. P . 
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(52) 



(53) 



(54) 



I Xil + I Xip 



xil + Xip S 1. 



Xip 6 (0.1) , 



|Wj|xijl + |Wj|xijp. 

j = 1. 
p = 2. 



p = 2.. 
1 = 1.. 

p = 1.. 
i = 1.. 



. Jvf: 

. P 

. P: 

. I , 

. P: 

. I , 



(55) 



Yr. 7s i 0- 



The above model for simultaneous test construction Is 
not very practical, because the numbers of constraints and 
variables are so large that the amount of CPU-time and 
conqputer memory needed for solving the model may be 
prohibitive (see Ezaaples). However, it may be possible to 
make use of the special structure of the model and develop 
heuristics that are capable of solving the model under 
realistic conditions. 

Multi-stage Testing 

Multi-stage testing differs from two-stage testing in 
that more than one subtest is administered after the routing 
test. The choice of each subtest depends on the scores on the 
preceding test. The linear pro^ amming models for the 



P3 



Two-Stage tests 
19 



construction of second— stage tests with constraints at 
subtest level can also be used for the construction of anilti- 
stage tests, provided that one modification is taken into 
account. In this case, set 0 in the constraint in (24) anist 
be redefined as the set of items selected for the preceding 
subtests. The models with constraints specified at test level 
can also be modified in a straight-forward way. so that they 
are useful for the construction of multi-stage test 
procedures. Extension of the model to simultaneous test 
construction is also possible; but will enlairge the problems 
associated with large nuodbers of variables and constraints. 

Examples 

In this section two examples are given. In the first 
example a two-stage test is constructed sequentially with the 
demands specified at test level. In the second example the 
two-stage test is constructed simultaneously. A simulated 
item bank for French with 300 items that fitted the 3- 
parameter model (a^ - 0(0.5.1.5); b^ - U(-3.3); Cj^ « 0.2) was 
used for both exaooples. The item bank was partitioned with 
respect to its content in a vocabulary (items 1-100). a 
grammar (items 101-200). and a reading comprehension part 
(items 201 - 300). The first 100 items of those subsets were 
of the multiple-choice type. The other items were of the 
matching type. 

The demands of the test constructor in both examples 

were: 
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- The ability leyels at which the target information function 
is specified for the routing test is s -1. 62 » 0. and 
63 = 1. 

- The second tests cure peaked at the ability levels —1 and 1. 

- The number of items in the routing test and second tests is 
20. 

- The examinee should answer in total less than 15 vocabulary 
exactly 12 grammar, and more than 15 reading comprehension 
items. 

- The number of multiple-choice items the examinee should 
answer is smaller than 25. For the matching items this 
nuinber is greater than 13. 

The linear programming models with 0-1 variables that 
were used in the examples can be solved by a branch-^and-^und 
method (Land & Doig. 1960). The models were solved on a DBC- 
2060 computer with a modified version of the program Lando 
(Center for Mathematics and Computer Science). The 
modifications in the branch— and-4x)und peu:t of the algorithm 
are described by Adema (1988). The CPO-times in the examples 
do not include the time needed for reading the input file* 
for the initialization and for writing to th<i output file. 
The CPO-times are shown to give an itnpression about the 
practicality of the approaches. 

EgflDplg 1 

In this example we consider the case of sequential test 
construction with demands at test level. 



Two-Stage tests 

21 

in LP model for constructing a routing test that 
fulfilled the demands was formulated. The routing test 
constructed with this model was coaqposed as follows: 7 
vocabulcury items; 4 grammar Items; 9 reading cotnprehenslon 
Items. 6 multiple-choice Items and 12 matching Items. The 
test Information value was 4.050 at 6]^. 5.809 at 62. and 
4.037 at 63. 

Given the composition of the routing test, the 
restrictions on the coinposltlon of the second tests were: 
" The second tests should contain less than 8 vocabulary. 

exactly 8 grammar, and more than 6 reading cooiprehexislon 

Items. 

- The second tests should contain less than 17 multiple- 
choice Items and more than 1 matching Item. 

The Information values of the second tests were: 
Second test at 6*^ = -1: 5.052; 
Second test at e*** = 1: 5.690. 

The total CFD-tlme for constructing the routing test and 

second tests was 7.3 seconds. 

Example 2 

In this example the routing test and second tests were 
cou^ true ted simultaneously with LP model (43) - (55). For the 
constructed routing test the test Information value was 3.689 
at O^. 5.608 at 62. and 3.742 at 63. The Information values 
of the second tests were: 

Second test at 6'*' = -1: 5.758; 

Second test at 6*^ = 1: 5.758. 
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The CPU-time needed for coosputing a solution was 659.3 
seconds . 

Discussion 

In this paper linear programming models for the 
construction of two-stage tesMng procedures are proposed. 
The model for simultaneously constructing the subtests has 
theoretical value as yet. because of the large number of 
(O.l)-variables and constraints in the model and the problem 
of specifying the ability levels of the second stage tests. 
However, it may be possible to develop heuristics for special 
cases. More promising were the models for sequential test 
construction. Especially the modol with constraints at 
subtest level is easy to apply, because the routing test and 
the second test can be constructed separately. When the test 
constructor specifies the constraints at test level, some 
problems will arise because we will have to take into account 
the construction of the second-stage te^t in the model for 
the routing test azKl reverse. These problems can be solved by 
introducing the constraints in (40) through (42). 

There are other forms of adaptive testing idiich may be 
more efficient than two-stage testing. The construction of 
such adaptive tests, using mathematical programming models is 
not always practical and can be too difficult if not 
impossible. As an example, the case of tailored te; cing is 
considered. In tailored testing one item is selected at a 
time so that the selection of the best item is easy: Select 
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the item that gives the most information at the current 
ability estimate of the examinee and also satisfies the 
practical constraints. After the selection of an item the 
practical constraints have to be adjusted, just like we 
adjusted the constraints in model (31)-(39) to the items 
«»^lected for the routing test. In fact, model (31)-(39) can 
be used for the construction of tailored tests by adapting 
the model In a straight-forward way and choosing ng s 1. 

Appendix 

In this appendix a simple algorithm is given for solving 
a model with objective function (23) and constraints (14) 
through (17) with subindex r replaced by s where the 
intersection of the subsets is empty. In the first part of 
the algorithm the items giving the most information at 

ability level 6*^ are selected from the subsets Ej(j 1 

Jg) and Gj(,^ s 1 Jq) such that the ^-constraints are 

satisfied and equality yields for the ^-constraints. In the 
second part of the algorithm the resc of the n items are 

selected from the subsets Gj(j 1 Jq) and Kj(j ^ 1. 

.... J]^) such that the ^-constraints are satisfied and the 
information at ability level 6'*' is maximized. For 
convenience, subscript s is omitted in the following 
description of the algorithm. 
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Algorithm: 

Step 1: Sort the items in the siibsets Ei Ej^. 

Ojq in sequence of decreasing information at ability 
level 9*^. Go to Step 2. 

Step 2: The first nSj (J = 1 ^E) and hq^ (J = 1 

Jq) items from the subsets Ei Ej^, Gi Gj^ 

are selected for the test. Go to Step 3. 

Step 3: Put the items of Ii tjj. and the items of 

Gi Gjq that are not selected in a queue. Sort 

the items in the queue in sequence of decreasing 
information at ability level 6*^. Go to Step 4. 

Step 4: If n items are selected the algorithm stops. If less 
then n items are selected, then take the first item 
from the queue. Check whether the selection of this 
item is feasible for the ^ -constraints. If so, add 
the item to the test. Repeat Step 4. 

We can adapt the algorithm so that it can also solve other 
problems. For instance, it is easy to include constraints 
(19) with r replaced by s. The purpose of this appendix was 
Just to show that ' Is sometimes possible to solve a 0-1 LP 
model by a simple and efficient algorithm. 
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